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This small-scale, mixed-methods study aims to investigate academics’ understanding of formative 
and summative assessment methods and how assessment literacy impacts on their teaching 
methods. Six semi-structured interviews and a scrutiny of assessments provided the data and 
results suggest that while these academics understand summative assessment, they have a 
poorer awareness of the implementation of well-constructed formative assessment. While the 
academics were able to clearly articulate the perceived benefits to students from undertaking 
formative assessments, they were less able to identify potential benefits for themselves as 
educators, so these went largely unrealized. Opportunities therefore exist for tutors to 
utilize the outcomes of formative assessment to improve student performance, particularly 
around tutor-reflection to amend future learning and teaching approaches in line with the 
theory underpinning summative and formative assessment methods.The study highlights the 
importance of considering all stakeholders when thinking about assessment literacy. 
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Introduction 

This paper will reflect on findings from a small-scale qualitative study on formative assessment. 
The study was initially prompted by a university-wide structural change at the research site that 
resulted in a curriculum redesign across the whole institution. 

Mindful of poor National Student Survey (NSS) scores in assessment and feedback over 
a number of years, the institutional review of the curriculum included a key strategic aim that 
required that ‘formative assessment opportunities are designed into the curriculum so that 
students receive feedback before they encounter high stakes summative assessment’ (Review of 
the Academic Framework Guiding Principles, Internal Policy Document, 2012).This encouraged 
academics to rethink their assessment approach, and one popular method was to implement 
in-class or online multiple-choice tests as part of a formative assessment strategy to enable 
students to monitor their progress towards a final assessment. Online tests were popular, given 
their ability to provide effective, rapid feedback (Yorke, 2005; Bartram and Bailey, 2010) developed 
with relatively low cost and that they could be reused as needed (Armellini and Aiyegbayo, 2010), 
and although this approach carries a fixed overhead, it is better suited for a mass approach to 
higher education than more traditional methods (Yorke, 2005). 

The small-scale qualitative study was therefore designed to investigate how academics 
integrated multiple-choice tests, in particular those delivered online, into first-year core modules 
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in a large business faculty, and what impact they had on their formative and summative assessment 
strategies. Following a brief review of the literature on formative assessment, the findings of the 
study will be discussed and then reviewed in light of current thinking about assessment literacy. 


Background and context 

The changing nature of higher education in terms of raised expectations of assessment methods 
(Di Costa, 2010), larger classes, and more diverse populations (Yorke, 2005; Bartram and Bailey, 
2010) has led to an increase in research into innovative methods of assessment because educators 
have had to rethink their practices (Bartram and Bailey, 2010). With a move towards ensuring 
assessment for learning rather than only of learning (Wiliam, 201 I), assessment methods have 
undergone significant scrutiny, with a particular focus on the relationships between summative 
and formative methods of assessment. 

Summative assessment is widely agreed to be a form of evaluation that provides a judgement 
that encapsulates all the evidence up to a given point (Taras, 2005), usually the end of a teaching 
period (Wininger, 2005). It can have the purpose of providing a student with a grade (Bloom 
et a!., 1971), where learning is seen in terms of knowledge, skills, and attributes (Kibble, 201 I), 
thus enabling students to be differentiated among their peers (Wininger, 2005). Summative 
assessment is therefore a powerful tool, because through the provision of grades it ‘creates 
texts - marks, grade sheets, qualifications - that reify identity’ (Pryor and Crossouard, 2008: 8); 
for example, leading a learner to identify themselves as a first class or a 2.1 student. However, 
there are negative aspects of summative assessment, mostly regarding its nature of focusing too 
much on ‘How did I do?’ rather than ‘How am I doing?’ (Yorke, 2005; Kibble, 201 I). 

As with summative assessment, the notion of formative methods is not universally 
understood. Often, academics disagree about the key purpose of formative assessment (Bennett 
et o/., 20l2;Taras, 2010; Wiliam, 2011) and it can be seen as an ‘add-on’ to regular teaching and 
learning methods rather than as an integral part of the overall strategy (Yorke, 2005; Hargreaves, 
2005). 

The definition of formative assessment has been through several iterations since the seminal 
work of Black and Wiliam in 1998. They propose a more recent definition (2009) that focuses 
on evidence coming from classroom practices rather than assessment yet retains the multi¬ 
stakeholder perspective, including both learners and teachers, and the emphasis that decisions 
made on the basis of that evidence should advance the stakeholder’s position in teaching and/ 
or learning: 

Practice in a classroom is formative to the extent that evidence about student achievement is 
elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the 
next steps in instruction that are likely to be better, or better founded, than the decisions they 
would have taken in the absence of the evidence that was elicited. 

(Black and Wiliam, 2009:7) 

Nicol and Macfarlane-Dick (2006) constructed a list of good feedback practices, highlighting 
seven principles to help encourage students to proactively engage in monitoring their own 
learning through formative assessment strategies. The list emphasizes feedback that helps 
clarify good performance and facilitates self-reflection through high-quality information about 
students’ learning that positively impacts upon motivation and self-efficacy. As well as these 
explicit student-oriented benefits, their list reiterates Black and Wiliam’s (1998; 2009) multiple- 
stakeholder-oriented definition that feedback should shape both teaching and learning practices 
through encouraging tutor-peer dialogue around learning and also provide information to tutors 
to support teaching practices and improve instruction. 
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Academic rationale for the study 

The original study was intended to be a small-scale scoping study with the objective of investigating 
how academics responded to the institutional requirement of embedding formative assessment 
in their modules with the aim of supporting summative assessment performance.Three key areas 
were identified for exploration through qualitative interviews, which were developed around the 
central themes of Black and Wiliam’s (2009) definition of using formative assessment to impact 
upon summative assessment through the collection and feedback of evidence or data: 

1. What do academics understand by the terms ‘formative’ and ‘summative’ assessment? 

2. How do academics conceive of the interaction between formative and summative 
assessment? 

3. How do academics manage the feedback process with formative and summative 
assessments? 

Six semi-structured interviews were held with module leaders who teach on a large first-year 
general business programme. An exclusive focus on the first year was adopted, given that the 
way teachers ‘approach their teaching of a general first year course may be quite different to 
their approach to teaching later year courses’ (Trigwell e£ a/., 1994: 77) and because it has the 
greatest potential impact on retention (Yorke, 2005). 

The nature of the programme meant that a wide range of disciplines was covered, which 
helped minimize potential single-subject bias. Rather, the breadth of disciplines covered enabled 
a range of opinions and experiences to be examined to see what variation of conceptions and 
approaches exist. 


Reflections on findings 

It was clear that our participants had good understanding of summative assessment as providing 
a grade, and formative assessment as enabling students to assess themselves and get feedback 
on their progress during the module. As the participants typically taught large modules, with an 
average registration of 420 students among the sample, the online assessments were seen as a 
way to provide feedback to a large cohort quickly and easily. 

These general conceptions were in line with the definition in the literature in relation to 
students, but were found to be limited when thinking more holistically about other stakeholder 
benefits. All participants held firm that the formative assessment was put in place for the benefit 
of students, and as such it was their responsibility to engage with the assessment, get the feedback, 
and make use of it. However, a key finding of the study was that when prompted, no participant 
commented on any other potential benefits, even for their own teaching practices. There was 
no formal or structured way in which the formative assessment results were reviewed, which 
differed significantly from the processes adopted when considering summative results. This 
meant that the part of the formative assessment definition that commented on the benefits 
for educators from using classroom evidence to make better instructional decisions (Black and 
Wiliam, 2009) went largely unrealized. 

On reflection, and with further examination of the interview transcripts, barriers to academic 
self-reflection became evident, which included time and lack of resources as the most common 
barriers. To illustrate this, when asked directly about monitoring the formative assessment 
outcomes, responses included: 

Online quizzes are not monitored.There’s an overhead in setting it up and the staff time is proving 

too costly. It takes a lot of time. 
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The reason we use multiple choice is to try and compensate for the huge numbers we have. A 

major problem is with resourcing and I haven’t got time to do it. 

From an institutional point of view, we have reflected that while the aim of integrating formative 
assessment opportunities into the curriculum through the review of the academic framework 
was successful, at present they largely remain just that - opportunities. Until there is a systematic 
review by both students and staff on the outcomes of the assessments, it is hard to make the 
case that both groups are making better educational decisions than would have been made 
without the assessments, and therefore our findings question whether the assessments are 
in fact formative in nature. The challenge therefore becomes one of facilitating longer-term 
engagement with the institutional strategic aim to take the use of formative assessment beyond 
implementation and into full integration in assessment strategies, with regular reflection by all 
stakeholder groups. 

With regard to how to move forward, our intention is to use this pilot study as a springboard 
to encourage academics to review their assessment methods, both formative and summative, 
identify the data each provides, and consider them in tandem. Fostering an environment where 
academics feel able to properly reflect on formative evidence will help improve pedagogic 
practices (Yorke, 2003) and reap the long-term benefits of improved pedagogic literacy (Price 
et a!., 2011). 

However, this is not a simple exercise.Yorke (2005) comments that the constraints experienced 
by academics, such as the emphasis on summative outcomes, increasing staff-student ratios, and 
the requirement for teachers also to be ‘research active’, mean that encouraging academics to 
change their practices is challenging and is‘threatening the use of formative assessment’ (Yorke, 
2005:486); our findings are consistent with this line of thought. 

This is clearly not making the most of the potential of formative assessment for improving 
learning and leads to the authors’ observation that simply making formative assessment part of 
an institutional requirement does not automatically lead to assessment for learning.With this in 
mind, we look to the literature on assessment literacy to reflect further on the findings. 


Assessment literacy 

Academics’ awareness and understanding of assessment and feedback can be considered part 
of their assessment literacy, defined as ‘the level of knowledge, skills, and understanding of 
assessment principles and practice’ required by a range of stakeholder groups (Taylor, 2009:24). 
This definition does not differentiate between the stakeholders, and indeed different authors 
focus on the various stakeholders in a variety of ways. Smith et al. (2013), for example, place 
emphasis on students, conceptualizing their assessment literacy as having three dimensions: to 
understand the purpose of assessment, awareness of the process of assessment, and the ability 
to judge their own responses and reflect on personal strengths and weaknesses. Popham (201 I) 
does not differentiate between stakeholders: ‘Assessment literacy consists of an individual’s 
understandings of the fundamental assessment concepts and procedures deemed likely to 
influence educational decisions’ (Popham, 2011: 267). However, the focus is firmly on educators, 
with the definition explored exclusively from their perspective, drawing on accountability 
and the need to ‘understand the nature of the instruments being employed to judge them as 
professionals’ (Popham, 2011: 269). The shift to a student focus does occur, but is still seen 
through the lens of the educator, and Popham highlights how diagnostic assessment can play a 
powerful role in student learning due to the ability to provide information on current levels of 
learning. However, this role for formative assessment in providing ‘instruction-enhancing tools’ 
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(Popham, 201 I: 271) is only possible if teachers have a good understanding of assessment, in 
other words if they have good assessment literacy. 


Reflecting on assessment literacy 

Undertaking this research, and the subsequent reflections on the findings, has highlighted 
the importance of considering all stakeholders when thinking about assessment literacy. For 
students, knowing the purpose and processes of assessment is crucial, as is understanding how 
to access and make use of feedback to improve subsequent work. However, the educator’s 
assessment literacy is equally important, and this has two significant aspects. Firstly, the educator 
needs to be able to develop meaningful formative assessment and associated feedback to ensure 
assessment for learning. Understanding the difference between assessment as an evaluation of 
student performance and assessment as a learning tool is crucial. However, an additional element 
in educators’ assessment literacy is to view formative assessment as a tool to improve their 
own practice.This was not evident in our own findings, where educators simply made available 
online tests and answers for optional use by students. Indeed, as noted by Yorke, while academics 
routinely reflect as part of their professional practice, it tends to be focused on their subject 
discipline rather than on pedagogic practice, and reflection on the ways that assessment practice 
and theory are integrated into the curriculum is underdeveloped (Yorke, 2003). By analysing 
online test results, it would be possible to identify areas of weakness with a view to improving 
learning in identified areas. This is in line with Black and Wiliam’s suggestion that feedback is 
‘used by teachers ... to make decisions about the next steps in instruction’ as well as by learners 
and their peers (Black and Wiliam, 2009: 7). 


Impact on practice and assessment literacy 

Overall, we found immense value in conducting this study, despite its small-scale nature. Although 
the sample was not large enough to enable any generalization, it did permit a valuable insight 
into academic practice at the research site. The findings prompted us to review the literature 
to examine why the full benefits of formative assessment went unfulfilled, leading us to consider 
assessment literacy as a possible lens, although the number of studies focusing on the academic 
perspective was startlingly small. Popham suggests this may be because most educators believe 
they already have a reasonable handle on what is meant when someone is identified as being 
‘assessment literate’ (201 I: 267). Indeed, it would seem reasonable to posit that any academic 
who has the authority to set student assessments should be fluent and fully literate about 
what the function, process, and outcomes of that assessment are. The literature clearly places 
the emphasis on academics encouraging students to develop their understanding of what an 
assessment means for them - it seems time that we, as academics, start to do the same. 


Conclusion 

Having reviewed the literature on assessment literacy, formative assessment, and assessment for 
learning, it is apparent that the focus is clearly dominated by the student perspective, benefit, 
and challenges. Institutions and educators need to realize the value of self-reflection on teaching 
practices for future benefit in the same way that we encourage and expect students to reflect 
on the feedback on their work; academics should practise what they preach. 
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